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PREFACE 


The  work  described  in  this  paper  was  performed  by  Dr.  Jun  Wang  of  the 
University  of  North  Dakota  and  Mr.  Shing  P.  Chu  of  the  U.  S.  Air  Force,  Armstrong 
Laboratory,  Logistics  Research  Division  (AL/HRG).  This  effort  was  conducted  under 
The  Air  Force  Office  of  Scientific  Research  (AFOSR),  Summer  Faculty  Research 
Program,  contract  number:  F49620-93-C-0063. 


SUMMARY 


In  recent  years,  several  techniques  have  been  developed  to  create  "intelligent" 
diagnostic  aiding  systems.  Most  of  these  systems,  including  the  current  Integrated 
Maintenance  Information  System  (IMIS)  diagnostic  module,  involve  modeling  the 
systems  to  be  maintained.  These  systems  have  the  disadvantage  of  requiring  extensive 
efforts  to  develop  them.  A  developing  technology,  neural  networks,  provides  a  promising 
alternative.  Neural  nets  develop  diagnostic  strategies  by  learning  from  past  experience 
with  the  system,  and  do  not  require  extensive  modeling.  Neural  networks  are  well  suited 
to  diagnostic  applications. 

This  paper  presents: 

-  An  aircraft  diagnostic  problem  formulated  as  mathematical  descriptions. 

-  A  detailed  description  of  constraint  construction. 

-  An  explanation  of  a  recurrent  neural  network  architecture  and  its  construction. 

-  Simulation  result. 


VI 


I.  ABSTRACT 

A  Bayesian  classifier  based  on  a  recurrent  neural  network  was  developed  for  aircraft  fault 
classification.  From  historical  maintenance  data,  the  posterior  probabilities  of  fault 
classification  based  on  given  fault  indicators  are  estimated  and  derived  using  the  Bayes’ 
rule.  Based  on  Bayesian  decision  theory,  the  fault  classification  problem  is  formulated  as 
a  linear  integer  programming  problem  to  minimize  an  expected  loss  function  using  the 
posterior  probabilities.  The  linear  integer  programming  problem  is  then  converted 
equivalently  to  a  standard  linear  programming  problem.  A  two-layer  recurrent  neural 
network  is  used  to  carry  out  the  computation  task  for  fault  classification  by  solving  the 
formulated  linear  programming  problem.  The  simulation  results  of  a  pilot  study  based  on 
the  synthetic  data  on  the  fire  control  radar  system  in  F-16  aircraft  show  that  the  neural 
network  approach  is  capable  of  real-time  aircraft  fault  classification. 

II.  INTRODUCTION 

An  aircraft  is  a  complex  electromechanical  system  composed  of  thousands  of  parts. 
Because  of  the  complexity,  aircraft  fault  diagnosis  and  classification  are  challenging 
tasks,  especially  for  military  aircraft  for  which  the  cost  and  time  are  considered.  The 
traditional  diagnostic  methods  for  aircraft  maintenance  using  technical  manuals  are  costly 
to  author  and  often  fail  to  isolate  the  cause  of  the  aircraft  failure,  thus  impacting  mission 
readiness  and  increasing  maintenance  costs.  High  field  maintenance  hours  are  often 
caused  by  incorrect  diagnoses  and  subsequent  false  removals.  In  addition,  historical 
information  from  Maintenance  Data  Collection  Systems  is  difficult  to  access  and  rarely 
used.  Each  of  these  factors  contribute  to  the  need  for  a  diagnostic  system  that  is  capable 
of  learning  from  historical  data  in  order  to  identify  the  faulty  units  and  correctly  predict 
the  nature  of  faults.  Artificial  neural  networks  provide  a  possible  solution  to  the  fault 
classification  problem. 

Resembling  biological  nerve  systems  more  or  less  in  structure,  neural  networks  are 
parallel  distributed  models  composed  of  many  simple  processing  elements.  In  processing 
information,  these  elements  operate  concurrently  and  collectively.  During  the  past  ten 
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years,  neural  networks  for  pattern  classification  have  been  one  of  the  most  active  areas  in 
intelligent  systems  research  and  various  neural  network  models  have  been  developed  for 
pattern  classification  (Specht,  1990).  The  results  of  numerous  studies  have  shown  the 
superior  performance  of  neural  networks  for  pattern  classification. 

In  this  paper,  a  two-layer  recurrent  neural  network  is  presented  for  aircraft  fault 
classification.  The  background  information  about  the  aircraft  fault  classification  is  given 
in  Section  III.  Based  on  the  Bayesian  decision  theory,  the  aircraft  fault  classification 
problem  is  formulated  as  a  linear  integer  programming  problem  and  then  converted  to  a 
linear  programming  problem  in  Section  IV.  The  energy  function,  dynamical  equation, 
and  architecture  of  the  recurrent  neural  network  are  discussed  in  Section  V .  The 
simulation  result  of  a  pilot  study  for  fault  classification  of  F-16  radar  systems  are 
presented  in  Section  VI.  Finally,  conclusions  are  presented  in  Section  VII. 

III.  BACKGROUND 

A  bench  check  serviceable  (BCS)  occurs  when  the  reported  faulty  component 
checks  good  when  tested  in  the  back  shop.  BCS  occurs  at  the  maintenance  shop  level  and 
is  always  the  result  of  unnecessary  removal  of  LRU.  Early  knowledge  that  an  LRU  has  a 
history  of  frequent  BCSs  and  is  likely  to  BCS  can  help  the  maintenance  technician  to 
modify  his  diagnostic  procedures,  thus  reducing  unnecessary  removal  of  LRU's.  A  well- 
trained  technician  can  tell  from  the  maintenance  history  of  the  LRU  whether  it  is  likely  to 
BCS,  thus  providing  this  project  with  a  set  of  sample  data.  This  report  presents  a  process 
for  developing  a  practical  fault  classifier  that  uses  neural  network  technology  and 
historical  data  to  identify  faults  in  today’s  military  aircraft  systems. 

IV.  PROBLEM  FORMULATION 

To  facilitate  the  ensuing  explanation,  let  the  numbers  of  part  units,  fault  categories,  and 
fault  indicators  be  denoted  as  m,  n,  and  p,  respectively.  Let  Xy  be  denoted  as  the  decision 
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variable  defined  as  follows:  for  i  =  1,  2, m,j  =1,2, ...,  n;  where  Xy  =  1  for  part  unit  i 
belongs  to  fault  category  j  and  zero  otherwise. 

From  past  data,  one  can  estimate  the  prior  probability  of  each  part  unit  (e.g.,  LRU,  SRU) 
belonging  to  each  category  and  the  conditional  probability  of  each  fault  indicator  (e.g., 
MFL)  for  any  part  unit  belonging  to  any  category.  From  past  data  or  the  prior  and 
conditional  probabilities  using  the  Bayes’  rule,  we  can  obtain  the  posterior  probability  of 
each  part  unit  belonging  to  each  category  given  any  fault  indicator.  The  prior 
probabilities  form  an  w  by  n  matrix.  The  conditional  and  posterior  probabilities  form  two 
mhy  nhyp  three-dimensional  data  arrays. 

For  each  misclassification,  there  is  always  some  associated  cost  penalizing  the  mistake. 
The  cost  can  be  in  terms  of  dollar  amount,  time  wasted,  or  a  combination  of  these  factors, 
e.g., 

lij  =  W]  Cij  W2  Tij  , 

where  Ijj,  Cij,  and  Tij  are  respectively  the  loss,  cost,  and  time  resulting  from 
misclassification  of  part  unit  /  in  fault  category  j;  w,  and  Wj  are  weighted  parameters  to 
balance  the  cost  and  time  criteria.  In  one  extreme  case  where  the  maintenance  time  is 
the  dominant  criterion  (usually  in  war  time),  w ,  =  0.  In  the  other  extreme  case  where  the 
maintenance  cost  is  the  dominant  criterion  (usually  in  peace  time),  w  2  =  0.  The  cost 
coefficients  form  an  m  by  n  cost  coefficient  matrix. 

Given  a  fault  indicator,  the  sum  of  the  Schur  product  of  the  loss  coefficient  matrix  and  the 
posterior  probability  matrix  corresponding  to  the  fault  indicator  constitutes  the  expected 
cost  (loss)  function.  By  minimizing  the  expected  loss  function  subject  to  some  feasibility 
constraints,  faults  can  be  classified. 

One  fundamental  feasibility  constraint  that  ensures  an  LRU  can  be  assigned  to  only  one 

n 

class  is  as  follows:  ^xu  =  1,'  i=  \,2, ...,  m. 

/=i 
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Given  limited  resources  for  expenditure  and  manpower,  another  feasibility  constraint  can 

m 

be  added:  <  /// ,  for  i=l,2,...,n;  where  Pj  €  {0, 1,  2, m)  is  the  upper  bound  of 

/=1 

numbers  of  part  units  allowed  in  category  j.  These  upper  bounds  allow  one  to  control  the 
classification.  They  can  be  determined  based  on  prior  information  on  the  size  of  the 
categories  or  the  maintenance  capacity  of  the  workforce. 

The  last  fundamental  constraint  is  the  integrity  constraint  defining  the  binary  nature  of  the 
decision  variables:  Xy  e  {0, 1}. 

In  summary,  the  fault  classification  problem  can  be  formulated  as  the  following  linear 
programming  problem:  For  a  given  MFLy^,  k=  \,2, ...  ,p; 


in  n 


maximize  ^  ^  Pot  kxy ; 

1=1  7=1 

n 

subject  to  ^X/7  =  l, 

7=1 

i  =  1,  2, ...  w; 

(1) 

VI 

j  1 »  2, ...,  n. 

(2) 

X 

m 

o 

(3) 

where  hj  =  w\  Gj  +  W2  Tij  . 


The  inequality  constraint  (2)  can  be  easily  converted  to  an  equality  constraint  by  adding  a 
slack  variable  >-j  in  for  each  j.  Because  the  coefficients  of  the  constraints  (1)  and  (2)  have 
the  total  unimodular  property  (Wang,  1994),  the  integrity  constraint  can  be  replaced  with 
the  non-negativity  constraint.  The  above  linear  integer  programming  problem  can  thus  be 
reformulated  as  the  following  linear  programming  problem:  Given  an  MFL^ , 
k=  1,  2, 


III  n 

maximize  ^  ^  Pyk  hjk  xy ; 

/•=!  7=1 

(4) 

n 

subject  to  Xij  =  1 , 

7=1 

11 

s 

(5) 

4 


j=  1,2, 


(6) 


Y^Xij  +  jui yj  ^  ^ , 


/=! 


x/,  >0,  ;;y>0,z=l,2,...,w;  7  =1,2, (7) 

Note  that  constraints  (5-7)  together  imply  xy  <  1  and;^-  <1. 

V.  DYNAMICAL  EQUATION 

To  solve  the  optimization  problem,  a  two-layer  recurrent  neural  network  is  developed 
tailored  from  the  deterministic  annealing  neural  network  for  convex  programming 
(Wang,  1994).  Based  on  the  formulated  linear  programming  problem  (4-7),  a 
computational  energy  function  can  be  defined  as  follows: 

j  f  n  \  ”  f  ^  " 

—  ^ ^ S  ^ f^'iy'  ~  0  ~  S ’’’  ’ 

^  /=!  V  7=l  )  ^  7=1  V /^]  ) 

For  simplicity,  the  same  symbols  are  used  hereafter  to  denote  decision/slack  variables  and 
corresponding  state  variables. 


The  dynamical  equation  of  a  deterministic  annealing  neural  network  can  be  defined  as  a 
gradient  system  based  on  the  energy  function: 

duij  _  SE{x,y,t)  dvj  _  dE{x,y,t) 


dt 


dXit 


dt 

Specifically,  the  dynamical  equation  of  the  neural  network  is  shown  as  follows: 

dUij 

dt 

dvj 
dt 


—  XiJ  —  ^  Xij  —  fj,iyi  +  1  +  7^  +  Pijk  lijk  € 


./=! 


;  =  1 


ffl 


/=l 


The  standard  unipolar  sigmoid  activation  function  is  used: 
x,j  =  giuij)  =  - — ,  yi  =  g(yj)  =  ^ 


1  +  e  l  +  e 

where  ^  is  a  scaling  constant  determining  the  sensitivity  of  activation. 
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The  architecture  of  the  two-layer  neural  network  is  shown  in  Figure  1,  where  the  state 
variables  of  output  neurons  and  hidden  neurons  correspond  to  the  decision  variables  Xy 
and  slack  variables  jj,  respectively. 


Since  ^  ^ ^  =  M?)  and  ^  =  &(z)g(l  -  e) ,  the  dynamical 


dt 


dwj  dt  dt  dvj  dt 


dz 


equation  can  be  rewritten  without  uij  and  vj  as  follows:  for 


dxij 

dt 


f 

) 

V7= 


n  rn 

-  =  -  Xij)  ^XiJ  +  ^XiJ  +  jUi-  P  hik  e 


-  T/)  X  ^'/  +  ;  T/  -  ^^j 
V./=1 


y 


Figure  1.  Two  layers  recurrent  neural  network. 


VI.  SIMULATION  RESULT 

While  the  proposed  neural  network  approach  can  be  used  for  general  classification  tasks 
with  any  numbers,  patterns,  categories,  and  evidence,  this  section  discusses  the 
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simulation  results  of  a  specific  pilot  study.  In  this  experiment,  the  task  is  to  classify  the 
line  replaceable  units  into  three  categories:  replaceable  LRUs,  repairable  LRUs,  and 
bench  check  serviceable  LRUs.  A  bench  check  serviceable  LRU  is  identified  as  the  false 
removal  of  the  LRU.  A  MATLAB  program  has  been  developed  for  simulating  the 
recurrent  neural  network.  The  testing  data  was  created  to  simulate  the  posterior 
probabilities  and  cost  coefficients.  The  network  was  then  tested  with  an  arbitrary  initial 
condition,  xq  =  0.5*ones(8*3,  1).  ^  and  p  are  selected  to  be  100  and  1,  respectively.  The 
result  showed  the  network  converged  to  the  optimal  solution  that  satisfied  constraints  (5- 
7)  in  about  500  iterations. 


VII.  CONCLUDING  REMARKS 

A  Bayesian  classifier  based  on  a  recurrent  neural  network  is  developed  for  aircraft  fault 
classification.  The  Bayesian  classifier  minimizes  the  expected  loss,  taking  both 
maintenance  cost  and  time  into  consideration.  The  proposed  recurrent  neural  network 
provides  a  parallel  computational  model  to  carry  out  the  optimization  task.  The  proposed 
Bayesian  classifier  can  serve  as  a  core  in  a  maintenance  decision  support  system  for 
aircraft  diagnosis. 
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